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We study the star lattice with Rashba spin-orbit couphng and exchange field and find that there 
exists the quantum anomalous Hall elTect in this system and there are five energy gaps at Dirac 
points and quadratic band crossing points. We calculate the Berry curvature distribution and obtain 
the Hall conductivity (Chern number v) quantized as integers, and find that v — —1,2,1,1,2, 
respectively, when the Fermi level lies in these five gaps. Our model can be view as a general 
quantum anomalous Hall system and, in limit cases, can give what the honeycomb lattice and 
kagome lattice gave. We also find there exists a nearly flat band with i/ — 1 which may provide an 
opportunity to realize the fractional quantum anomalous Hall effect. Finally, the chiral edge states 
on a zigzag star lattice are given numerically to conflrm the topological property of this system. 
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Introduction. -T}\e quantum Hall effect was observed 
[I'l in 1980 in tvifo dimensional electron system, in vifhich 
Hall conductivity takes quantized value Gxy — ve^/h. 
The integer value v is called the TKNN number [2j or 
Chern number. The essential ingredient of Hall effect 
is to break the time-reversal symmetry of the system. 
Thus, introducing an external magnetic field is not the 
only way to produce this effect. In fact, anomalous Hall 
conductivity had been observed in ferromagnetic iron and 
ferromagnetic conductors since 1881 [?>\. The internal 
magnetization plays an essential role in this so called the 
anomalous Hall effect. It's naturally to ask if it is possible 
to produce the quantum anomalous Hall (QAH) effect 
without external magnetic field. 

A model constructed by Haldane demonstrated that in- 
tegral quantum Hall (IQH) effect can be realized without 
the Landau level induced by the external magnetic field 
[j|. In this spinless model on the honeycomb lattice, stag- 
gered magnetic field was introduced to break the time- 
reversal symmetry, while the magnetic flux per unit cell 
is zero so that no Landau levels present. This staggered 
magnetic field, in other words, the complex next near- 
est hopping amplitude opens a gap at the Dirac point. 
At half fining, when the Fermi level lies in this gap, the 
Chern number of this system takes the value v = ±1. Be- 
sides this toy model, several more realistic models which 
are based on various systems, have been proposed. For 
example, in Anderson insulators I5jl, HgMnTe quantum 
well [61, optical lattices ly 8| , magnetic topological insu- 
lators [0] , graphene [ifl, [III , kagome lattices [I4I . 

In this letter, we consider a star lattice model [see 
Fig. [Ija)] which has close geometry connection to the 
honeycomb lattice and kagome lattice [ij] and study its 
QAH effect. We calculate the Berry curvature in momen- 
tum space and find the Hall conductivity quantized as an 
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FIG. 1: (a) The lattice structure of the star lattice, where the 
bond length is a and oi, 02 are the primitive vectors of its 
Bravais lattice. The sites in the unit cell are labelled as 1 to 
6. The nearest hopping amplitude within triangles (all gray 
bonds) is t\, the nearest hopping amplitude between triangles 
(all black bonds) is ^2- (b) The reciprocal vectors (61 and 62) 
and the first Brillouin zone. 



integer value when the Fermi level lies in the topological 
nontrivial gap opened by Rashba spin-orbital coupling 
and exchange field. Unlike the proposals on honeycomb 
lattice and kagome lattice |1CH12| . five gaps are opened 
due to the lattice structure complexity of the star lattice. 
Thus, in our model, there exist five Hall plateaus when 
the Fermi level lies in these gaps and there is a nearly fiat 
band, with nonzero Chern number [u = 1), which may 
provide an opportunity to realize the fractional quantum 
anomalous Hall (FQAH) effect when electrons interaction 
is introduced. We also calculate on the zigzag ribbon and 
demonstrate that the excitation states are exactly local 
on the edge when the Fermi level lies in the bulk gap. 

Hamiltonian.-The star lattice can be seen as replacing 
the site of honeycomb lattice by triangle, or replacing 
the site of kagome lattice by segment. The number of 
sites per unit cell in star lattice is six, three times as in 
honeycomb and twice as in kagome. In this letter, we 
focus on the nearest hopping term and assume that the 



hopping amplitude takes the same value ii in triangle 
and ^2 between triangles (Fig. [T|). The nearest hopping 
tight-binding Hamiltonian is 



Ho 



E 

<i,j>,Q,A 



E 

<i.J>,a,A<r^A 



cLcja -t2 } , cj^Cja + h.C, 



. . (1) 

where a is the spin index, A means "within triangle" 
(all the gray bonds in Fig. W^), A ^-s- A means "between 
triangles" (all the black bonds). 

By writing this Hamiltonian in momentum space 
and diagonalizing the Hamiltonian matrix H {k), the 
band structures are obtained, as shown in Fig. [5Ka)-(c). 
These band structures demonstrate the close connections 
among the star lattice, honeycomb lattice and kagome 
lattice. When ^2 < f^i, the probability for electrons 
to hop out of the triangle is smaller than that between 
triangles. This means that the three points are bound 
together strongly and can be seen as shrinking into one 
point. As a result, the low energy bands at this case look 
like graphene [see Fig. [21[a)]. In contrast, when ^2 > |ti, 
the low bands look like kagome lattice [see Fig.[2Kc)]. We 
also notice that at ^2 = f^i, the gap closes. 

As shown in Fig. [2l[a) , a band gap already exists when 
only considering the nearest hopping. However, this is 
not a topological gap. In order to open topological gaps, 
we introduce Rashba spin-orbit coupling and exchange 
field as follow 

Hrso = UrSO E ('^"/3 ^ ^U")4a'^j73 + ^■'^■^ (2^) 
<i,j>,a,l3 

(2b) 



Hx = Ay^cLc,:„cr; 



' / J ^ia lOi" acn 



where Irso is the strength of the spin-orbit coupling, a 
is the vector Pauli matrices in spin space, dij is the unit 
vector point from site j to i, and A is the strength of the 
exchange field. 

So, the total Hamiltonian is 



H = Ho + H_ 



RSO 



H, 



(3) 



The band evolution of this Hamiltonian is shown in 
Fig.[5J Without Rashba spin-orbit coupling and exchange 
field, the six-sites unit cell forms six bands which are dou- 
bly degenerated. With only Rashba spin-orbit coupling 
Hrso is added, the spin degeneracy is lifted, except at 
several k points. The exchange field alone takes the sim- 
ilar effect, but leaves degeneracy at different k points. 
And band gaps can be opened when both interactions 
are added into the hopping Hamiltonian. In following, 
we will confirm that these band gaps are topologically 
nontrivial and that they can realize the quantum anoma- 
lous Hall effect. 

Berry Curvature and Chern number.-The Berry cur- 
vature in crystal momentum space is an essential con- 
cept and should be included when considering solid state 
system [l4|. Here, the intrinsic Hall conductivity can be 







FIG. 2: (a) The band structures for the nearest hopping tight- 
binding Hamiltonian Ho with i2 ~ ti. (b) t2 = fii- (c) 
t2 = 2fi. (d-f) The band evolution for introducing Rashba 
spin-orbit coupling and exchange field when t2 = ti. (d) 
Only Hrso is introduced, the strength of spin-orbit coupling 
is taken to be tRso = 0.2fi. (e) Only H\ is introduced, the 
strength of the exchange field is taken to be A = 0.2fi. (f) 
Both Hrso and H\ are introduced. The parameters are taken 
to be t2=ti, tRso = 0.2ti, A = 0.2fi. 



written as the summation of Berry curvature of all bands 
under Fermi level llSlI as follow 



e' 2n 



''^'y^TWr; E ^-iEn,k), 



h NV 



(4) 



k,E„<EF 



where N is the number of primitive unit cells, V is the 
volume of the unit cell. The nth band's Berry curvature 
nz{En, k) can be obtained by 



^z{En,k) 

- E ■ 



-2lTi\{ll}nk\dH {k) /dkx\'4>rnk){'4'mk\dH {k) /dky\i}nk) 



{E„ - Emf 



The Chern number is given by 

27r 



= ^ E ^^(^-^) 



NV 



k,E„{<EF) 



h E Jl/kn.iE.,k)., 






(5) 



(6) 



(7) 



when N — > oo. 

Writing the Hamiltonian ([3]) in momentum space and 
calculating the Hall conductivity by Eq. (|3]), we get the 
Hall conductivity a^y as a function of Fermi level Ep^ as 
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FIG. 3: (a) The Hall conductivity (or Chern number) as a 
function of Fermi level Ef with i2 = ti, tRso = 0.2ii, A — 
0.2fi. (b) The corresponding density of states (DOS). 




FIG. 4: The total Berry curvature distribution in momentum 
space Qz (k) = X]_b i<e ) ^^ (En, k) for the star lattice with 
i2 = t\, tRso = 0.2ii, A — 0.2ti, when Fermi level lies at 
(a) Ef = -2.77. (b) Ef = -2.32. (c) Ef = -0.21. (d) 
Ef = 0.36. (e) Ef = 1.27. (f) Ef = 1.0 



shown in Fig. [Sja) . Five Hall plateaus are obtained when 
Fermi level lies in the gaps (as suggested by the density 
of states shown in Fig. Eljb)). The Chern number take 
quantized value 1/ = — 1,2, 1,1, 2, respectively. 

The two ly = 2 plateaus come from the gaps opened at 
Diracpoints, which are similar to that in honeycomb lat- 
tice [iO] and kagome lattice [iJl ■ The two v — 1 plateaus 
come from the gaps opened at quadratic band crossing 
points (QBCP), which are similar to the kagome lattice 
[12] . However, there is only one v = 1 plateau in kagome 
lattice. What's more, one of these nonzero Chern num- 
ber belongs to the nearly flat band. This novel prop- 
erty provides the great potential to realize the fractional 
quantum anomalous Hall effect in the star lattice. The 
V = —I plateau also comes from the gap opened at Dirac 
point. But this gap will disappear when |2- increase. The 
reason that v = —1 can appear in our model is that the 
bands of the star lattice (Fig. [5]) are narrower than those 
of honeycomb lattice and kagome lattice. This results in 
that a direct gap can be opened above the lowest band. 
So, when |2. increases, the bands become wider, the di- 
rect gap become an indirect gap, the Hall conductivity 
becomes not quantized. 

To further confirm the above interpretation, we plot 
the total Berry curvature of the bands below the Fermi 



level Q.z{k) = YIiE (<e )^z{En,k). When the Fermi 
level lies in the gaps, peaks at the minimal energy differ- 
ence points are clearly seen (Fig. H]). 

Chiral Edge States.-ThB nonzero Chern number also 
manifests as the presence of chiral edge states which lo- 
calize at edge and propagate in one direction. Now, we 
study the edge state property on the zigzag star lattice 
ribbon (Fig. [5]). Periodic boundary condition is taken 
along y direction (parallel to the edge). Open boundary 
condition is taken along x direction. The band structure 
of a 324-sites width ribbon is shown in Fig. [5] 

We can see that edge bands appear in topological gaps, 
and no edge bands present in the gap above the fourth 
band. The absence of edges bands in this gap confirms 
that it is a topological trivial gap, as mentioned. 

To further demonstrate the topological property of 
these gaps, we calculate the edge state functions corre- 
sponded to the points marked on the edge bands (Fig.[S]). 
The probability density quickly decrease as going into 
the bulk. The group velocity —g^p- suggests these edge 
states propagate in different directions at different edge 
(Fig. [21). So chiral edge states appear when Fermi level 
lies in the topological nontrival bulk gaps. The number 
of these edge states indicate the absolute value of Chern 
number. The chiral of the edge states indicate the sign 




FIG. 5: Illustration of a zigzag star lattice ribbon which is 
infinite in y direction and has width W in x direction. The 
unit cell is indicated by the dashed lines. 








FIG. 6: (Color online) The band structure of a W=324 width 
zigzag star lattice ribbon with i2 = ti, taso = 0.2ti, A = 
0.2ti. Edge bands appear in the topological nontrival gaps. 




FIG. 7: (Color online) The edge states probability for Fermi 
level indicated in Fig. [6] (green dashed lines), (a) Ef = 
— 0.21fi and ky = TTxTTTrTrT ^'-"" ^^^ point marked as A, 



1 _ Mil ■Jit 

'^y ^ 100 (2 + V3)a 



100 (2 + V3)a 

point D. 



100 (2+x/3)a 

for point B. (b) Ef = 
for the point marked as C, ky 



100 (2+^)a 



for 



of Chcrn number. 

Conclusion.~ln summary, we have studied the quan- 
tum anomalous Hall effect on star lattice in the presence 
of both Rashba spin-orbit effect and exchange field. The 
Chern number calculated from the Berry curvature in 
momentum space and the edge states calculated on the 
zigzag ribbon confirm the emergence of QAH effect when 
Fermi level lies in the topological gaps. Due to the com- 
plex lattice structure of star lattice, our results include 
what are obtained in honeycomb lattice which has Chern 
number v = 2 and kagome lattice which has Chern num- 
ber 1/ = 1 or 2. What's more, a nearly flat band with 
nonzero Chern number (v — 1) also appear in our model, 
which provide the great potential to realize the fractional 
quantum Hall effect without tuning the ration between 
nearest and next-nearest hopping amplitude |16l4l8{ . 

Finally, the physical realization of the star lattice 
would be not very difficult considering its two cousins, 
honeycomb lattice and kagome lattice, have been real- 
ized in optical lattices [19, 20]. We also notice that the 
spin-orbit coupling has also been realized in optical lat- 
tice [2l|. Both of these offer the possibility to realize 



QAH effect in the star optical lattice. In solid state sys- 
tem, a ploymeric iron(III) acetate with underlying star 
lattice has also been reported |22l |. 
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